Security video monitoring client

ABSTRACT

A security video monitoring client system and method of presenting feeds within the same are disclosed. Out-of-process renderer components executing outside the client program provide an important performance advantage, particularly as to the number of video feeds to decode and/or render that can be decoded. This external thread execution strategy likewise facilitates fault isolation, shielding the client from undesired effects arising from the termination, restarting, and/or any other action to be taken as a result of anomalous operation of any one or more out-of-process renderer components.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present patent application is a continuation of U.S. patentapplication Ser. No. 16/431,417 filed on Jun. 4, 2019, that is acontinuation of U.S. patent application Ser. No. 14/795,303 filed onJul. 9, 2015, the contents of which are hereby incorporated byreference.

TECHNICAL FIELD

The present relates to security software and more particularly to thepresentation of multiple camera feeds in a security video monitoringsystem. The present also relates to managing the compositing of specificwindowed components within a graphical operating system's desktop. Morespecifically, it relates to rendering different visual elements,including software components, plugins, or video feeds, each runningwithin different operating system processes, while combining therelative appearance and operation of said components into a singleseemingly integrated GUI experience as seen from a human user. Thepresent also relates to handling said components subsequent toindependent fault isolation of the same.

BACKGROUND

Incorporation of existing elements into new entities is a recurringtheme in many areas of endeavor. Code reuse, whether specificallyinvolving templates, procedures, functions, libraries, components and insome cases entire software works, has been practiced in various formssince the dawn of computing. Warnings against reinventing the wheel arewell-known to engineers and software developers in general. Contemporaryreuse of software components lives on, particularly in the form ofadd-ons, plugins, drivers, and various open source initiatives, each ofthese existing elements offering myriad modes of encapsulation orintegration within new applications. More recently, the trend towardreuse in software contexts increased in popularity with theproliferation of open architectures that encourage the integration ofthird party components to operate within the framework some largerapplication. One characteristic of such component integration involvescoherent interaction between the integrated components and integratingapplications. In many cases, the incorporation is seamless, as theformer components, once integrated, appear to form a native part of agiven application.

Yet despite its many benefits, component reuse in general andintegration of third-party content in particular often carry with themthe original shortcomings of said component. Furthermore, the extent ofsaid shortcomings may be known in certain cases and accordingly workedaround. In other cases, these shortcomings remain largely or entirelyunknown and are only discovered following subsequent integration withinan application. The latter involves a transfer of risk and vulnerabilityto the party integrating or importing said components (in addition tousers of said resulting software), as the latter components' stabilityis not guaranteed. Relatedly, the often ambiguous nature and extent ofthe instability is an ill-defined characteristic. This flows from thefact that lack of stability in third party components undermines thestability of our client's application overall.

A related consideration to software component reuse is the specificparadigm by which said reused components are actually integrated. In thecase of applications that make use of numerous graphics and/or videocomponents, one important integration paradigm is that of compositingwindow elements. Compositing, also called composition, involvescombining various elements from different graphical sources orrespectively superposing them. Such sources may consist of the graphicaloutputs from one or more computer applications. A computer system'scompositing functionality, typically implemented via a compositingwindow manager disposed within said system's graphical section, aims togenerate a resulting output or “scene” comprising output from variousdisplay applications whose contents appear as though all said graphicalelements are part of the same scene.

Techniques to correctly determine the value of each pixel within adisplay require considerable processing resources. Notably, suchtechniques involve identifying the topmost element (i.e. the item havingthe highest Z-order value) among two or more overlapping two-dimensionalobjects, this process being somewhat iterative and complexified whenmultiple superposed transparent objects are present. Additionalresources are typically required if rapid changes associated with at anyone or more said graphical elements occur, with further resourcesrequired when alpha compositing considerations are relevant. Althoughexisting system-level compositing functionalities provide to eachapplication its own compositing engine, said engines do not typicallyintercommunicate. This further causes said techniques to beresource-intensive, since such rapid changes are typically associatedwith increased display activity. Accordingly, composition for regions ofpixels where such increased display activity takes place must be donemore frequently than for those regions—or even overall displays—whereless or no activity occurs. The need for increased composition activitytypically results in increased demand placed on the responsiblecompositing window manager. Of course, to satisfy such increased demandfor composition, a related increase in resource requirements isnecessary. The absence of such resources result in a limited compositingcapability of a computer system's display, with negative repercussionson performance and overall user experience using one or moreapplications on said computer system. To avoid such performance losses,dedicated handling of compositing functionality by graphics hardware hasbecome commonplace. In certain cases, some technologies use screenscraping in order to mix different rendering technologies, however thesetechniques lack the efficiency required for video streams.

A noteworthy case in which composition capability is called for occurswhen a considerable portion of a display area is occupied by a videowindow. Superposition of additional visual entities atop said videowindow, such as digital on-screen graphics (e.g. for branding oridentification purposes), or user interface component objects (such asplayback controls) may be imagined. The result of compositing istypically necessary for each pixel within a display, and any subsequenttransparency or alpha channel blending atop it needs to be done as well.This is why ideally, such compositing is typically left to the graphicssection—typically a dedicated graphics card. While such “stacking” ofgraphical entities (e.g. windowed applications) might be desirable forenhanced functionality, it likewise makes a system correspondinglyvulnerable to the shortcomings of composition arising from the lack ofcommunication between of respective compositing engines and the (windowsand/or graphical elements of) various applications.

In the case of an application comprising an array of video feedsreceived from a network of security cameras, an intense amount ofprocessing is required for compositing each feed in addition to theadditional graphical information typically superposed upon eachrespective feed. The present state of the art limits the amount of videofeed data that may be composited and displayed with respectiveadditional visual entities atop said feeds, all while a computer systemretains an acceptable level of performance. In addition, the growingneed for both higher definition images and higher framerate within manyapplications in which lower quality video (such as security monitoring)was previously acceptable have made the status quo unacceptable,particularly in light of lacking inter-process communication betweencompositing managers.

Concurrent with the need to overcome, ensure, and maintain propercompositing functionality for high quality video applications is theneed for a video monitoring system to cooperate with possibly numerousthird-party software components, particularly for use in decoding andrendering graphical elements. Moreover, abnormal operation is arecurrent concern for all components within a system and said abnormaloperation is all the more a cause for concern in cases where keycomponents may be provided by third parties. A component reuse modeladmitting third party components introduces vulnerabilities resultingfrom component reuse may be minor or fairly innocuous. Yet in extremecases, one or more reused components may be ill-suited, malfunctioning,or incompatible with the application as a whole into which they areintegrated that the latter may crash, be impaired, and/or or terminatesuddenly with particularly devastating consequences.

Likewise, window compositing is known for being a particularlycompute-intensive operation. This is particularly the case when multipledesktop components contribute to the final image to be displayed.Computing the final color or shade of a pixel is likewise a constituentpart of the compositing process. Various schemes to improve theperformance of desktop compositing have been proposed but many of theseare subject to the vagaries of operating system implementations, whichtypically differ from version to version. Retained and immediate moderendering and composition capabilities are possible using variousschemes, both software and hardware. However, a problem experienced withcurrent client applications is that like all desktop applications,retained mode rendering prevails. Painting and repainting variousportions of a desktop—including attendant garbage collection—becomesprocessing intensive, as all pixels of an entire desktop must bevalidated for the correctness of their pixel values, composition andrendering engines must operate on all of said pixels, following whichsaid composition is issued to the graphics card. Such operationstypically bear an enormous processing cost and have a correspondinglynegative effect on user experience, blocking video decoding and playbackthreads typically present on the same process as a client application.

The isolation of software components both for purposes of encouragingcomponent modularity as well as for facilitating fault isolation indesign is known in specific fields of computing. However, overlappingtechnical challenges from the respective technologies of system-levelcomputing and user interface applications involved in displayinggraphical data and/or handling rapidly increasing volumes of video datahave prevented a contemporary extension of such principles to fieldssuch as video monitoring where they are needed. Such is the purpose ofthe present application.

SUMMARY

The present invention may typically be implemented on any of severalplatforms, including a security video monitoring workstation, one orseveral computers, and/or homologous processing devices.

Persons skilled in the art will be familiar with the notion of aZ-order; with reference to a graphical application, this notiontypically refers to a precedence policy indicating, from a given vantagepoint, which items may—in whole or in part—appear closest or furthestfrom said vantage point. Said persons will likewise appreciate that, inthe case of overlapping items, the notion may alternatively refer towhich may item(s) appear to be superposed atop and therefore obscure theother(s).

Transparency as discussed herein is to be appreciated as a conceptcomplementary to that of Z-order. Transparency refers to the genericquality of a window layer to let through, rather than block, thegraphical content or attributes belonging to one or more window layers,as well as the circumstances under which said qualities are present.When such window layers have an inferior Z-order, their content orgeneral graphical attributes are obscured by part or all of a graphicalelement located superjacent to them. Accordingly, semi-transparency mayrefer to imparting a measure of said transparency to an entire windowlayer while retaining a certain level of opacity; alternatively,semi-transparency may consist of providing delimited portions of awindow with transparency while other portions of a window layer areentirely opaque.

According to a first aspect of the present invention, a window moduleinstantiates and manages a graphical user interface comprising asemi-transparent window structure over at least a portion of the totalarea of said window structure. Said window structure includes an arrayof tiles, with each tile having a particular dimension within thesemi-transparent region of said structure. An arraying module furtherpositions and dimensions the visual outputs provided by one or moreout-of-process renderer components to said window structure. Therenderer components are said to be out-of-process because they performtheir rendering tasks in one or more processes external to that of theclient system described herein. Upon an electronic visual display, thevisual outputs produced by said renderer components are presentedsubjacent to said window parent structure, said visual outputs placed insuch manner as to line up each subjacent to a respective one of theaforementioned tiles, whose transparent quality allows a human user toview the visual outputs. An operations setup module causes the renderingof said plurality of security camera feeds by coordinating such tasks asinitializing out-of-process renderer components to display any one ormore said visual outputs, and by restarting any one of saidout-of-process renderer components should an error or problem withrendering operation be encountered.

According to a second aspect of the present invention, a method ofpresenting a plurality of security camera feeds within a security videomonitoring client system is described. This method includes implementinga semi-transparent window parent structure as discussed above,comprising an array of transparent tiles, with said window parentstructure having particular dimensions. The method also includespositioning and dimensioning the visual outputs from one or more processrenderer components subjacently the parent window structure so that eachthe visual outputs lines up with a transparent tile. Finally, the methodincludes rendering the security camera feeds by one or moreout-of-process renderer components and restarting any one of theserenderer components should an error or problem be encountered.

According to a third aspect of the present invention, a tangiblemachine-readable storage medium to manage and implement the operation ofthe second aspect as described is disclosed.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be better understood by way of the following detaileddescription of embodiments of the invention with reference to theappended drawings, in which:

FIG. 1 shows a structural layout and interconnections view of thevarious modules comprising an embodiment of a prior art embodiment inwhich renderer and non renderer components, as well as their respectivevisual outputs and visual elements operate within the same process asthe client program of said prior art embodiment.

FIG. 2 shows the conceptual relationship of a prior art embodiment inwhich an in-process renderer's visual element is laid out superjacentlyto an in-process renderer component's visual element and output within atiled program window.

FIG. 3 shows a structural layout and interconnections view of thevarious modules comprising an embodiment of the client program,including external components operating in another process or executionthread.

FIG. 4a shows an exploded conceptual view of the output of anembodiment's window module as output to a display, showing the relativez-order of a semi-transparent window parent structure, superjacent to anarray of transparent tiles, subjacent to one of which tiles are shownvisual elements of a non-renderer component, subjacent to which is shownan example visual output of an out-of-process renderer component.

FIG. 4b shows a screenshot of an example embodiment's window module thatimplements right-to-left text directionality.

FIG. 5a shows the conceptual relationship between a semi-transparentwindow parent structure and an array of transparent tiles, said tilesillustrated as subjacent to said window parent structure.

FIG. 5b shows a screenshot of the panes, popups, and other GUI featuresand visual elements that in an example embodiment may appear togetherwith the semi-transparent window parent structure and array oftransparent tiles, some of said GUI features and visual elementsappearing superjacent to video feeds.

FIG. 6 shows an exploded view of an embodiment's program windowreceiving a user point-and-click input upon a transparent tile containedwithin a semi-transparent window parent structure, said window parentstructure superjacent to visual elements of non-renderer components,which are in turn superjacent to the visual output of an out-of-processrenderer component.

FIG. 7 shows a magnified partial view of an embodiment's program windowwith emphasis on a single transparent tile comprising with visualelements and visual output composited within a semi-transparent windowparent structure.

FIG. 8 shows an example depiction of an array of transparent tilesdisposed within a semi-transparent window parent structure, with many ofsaid transparent tiles containing various non-renderer components'visual elements superposed.

FIG. 9 shows an abstract collaboration between a semi-transparent windowparent structure and a series of transparent tiles upon many of whichvarious non-renderer components' visual elements are disposed.

FIG. 10a shows an abstract collaboration between a series of transparenttiles and a semi-transparent window parent structure upon which variousnon-renderer components' visual elements are disposed.

FIG. 10b shows a screenshot depicting the series of transparent tilesand the semi-transparent window parent structure in FIG. 10a in anexample embodiment.

FIG. 11 shows a detailed view of the relationship between video feeds,capture device such as a camera, and a scene to capture with respect tothe remainder of the module structure of the client program.

DETAILED DESCRIPTION

With reference to FIG. 1, comparable video monitoring systems known inthe art consist of a client program 100 that receives video feeds 101from one or more external sources, such as one or more cameras. Saidprogram 100 typically blends elements such as user interface controls,on-screen graphics, and other visual elements 103 received fromnon-renderer components 102 running within the client program 100process. Moreover, visual outputs 105 corresponding to the scenescaptured through the video feeds 101 are received from renderercomponents 104 likewise operating in-process. These visual outputs 105and visual elements 103 are typically received by an output manager 107which amalgamates the foregoing 103, 105, implementing the layout ofvarious tiles to composite in accordance with the requirements of theclient program 100, before outputting the result to a compositionmanager 106, typically located externally to the client program 100, foractual composition. Said composition manager 106 handles the compositingduties required to composite the visual elements 103 atop the visualoutputs 105 as required by the client program 100, in some cases foreach one of a number of individual tiles (FIG. 2) corresponding to anindividual video feed 101.

Persons skilled in the art will readily appreciate that in a computingcontext, the term “client” refers to an application that makes use ofobjects provided by another component. Accordingly, the location of acomponent is an essential element in describing said component withrespect to a client. Thus, a component is said to operate “in-process”if it runs in the same process as a client, and “out-of-process” when itruns alone within a dedicated process. Furthermore, an out-of-processcomponent has the benefit of possessing its own execution thread,whereas an in-process component uses the client's execution thread. As aresult, the decision to run in-process or out-of-process may in certainscenarios be informed in part by design, implementation, and/or runtimeconsiderations. For instance, if a component's operation is by its verynature likely to slow down or otherwise impair the client's executionthread, said component is likely best run out-of-process. Alternatively,if a component does not considerably impact the operation of the client,then it may run in-process and benefit from the fact that notransmission across processes need take place for it to properlyexecute. Additionally, a given application with one bitness (e.g. 64bits) may require but be unable to load a much-needed component with adifferent bitness (e.g. 32 bits). Segregating the operation of certaincomponents within distinct processes may thus likewise be necessary incases where disparities exist in the respective bitnesses of multiplecomponents that are otherwise expected to interoperate. Theseconsiderations play an important role in embodiments disclosed herein.

With reference to FIG. 3, embodiments typically involve thepresentation, on an electronic visual display 208, of several videofeeds 201 received from security cameras 401 or other imaging sources.The video monitoring client system of which various embodiments aredescribed herein typically accepts user input 210, 211 for controllingaspects of the display of said feeds 201.

Embodiments of the present security video monitoring client systemcontemplated herein may be implemented and/or deployed on one or severalcomputers, monitoring workstations, and/or homologous processors.Likewise, embodiments comprise the modules enumerated herein and whoseoperation and interoperation will be described in greater detail in thevarious sections.

The description below is intended to non-limitingly illustrate variousembodiments of the present invention.

Input and Event Listener Module

In various embodiments of the present invention, the event listenermodule 215 receives directions from one or more human users who may viewvideo feeds 201 through an electronic visual display 208. Said display208 may in various embodiments be a liquid crystal display, a TV screenor computer monitor. In other embodiments, an input source, such as amouse, a touch screen, or other technology implementing some other humaninput 210 device, may be integrated to or separate from said electronicvisual display. In another embodiment, the event listener module 215 maybe partially implemented across multiple modules, including thosefurther described herein, such as partially in a visual element 203 of anon-renderer component 202 (such as a GUI playback interface element) ona([n] especially tactile) display 208 and/or partially within the clientprogram 200 itself.

Instructions received from one or more human users are issued toembodiments. Accordingly, in various embodiments, the event listenermodule 215 receives two broad categories of human instructions. Onecategory of said instructions consists of user point/click input 210(typically for selecting a specific visual element 203 belonging to aspecific tile), whereas another category is user repositioning input 211(such as for specifying the dimensions or any other attribute of a tile,of the window parent, or of the disposition of a tile). A human-suppliedinstruction may comprise one or both of 210 and 211.

User point/click input 210 refers to interactions with the videomonitoring client system that consist of using a human interface device,such as a pointing device which a human user may physically manipulate,to specify positions within the window parent of said video monitoringclient. In an embodiment, these positions may consist of (x,y)coordinates of a correspondingly two-dimensional parent window (furtherdescribed herein) comprising an embodiment of the client program 200,260.

User repositioning input 210 may be further distinguished as two typesof interactions. Repositioning may refer to an allocation (orreallocation) action wherein the tile location that a feed should occupyand within which said feed should be displayed is specified (oralternatively, re-specified or reassigned) by a user. This isdistinguished from repositioning which may involve moving the locationof said parent window structure about the electronic visual display 208on which it is shown. In some embodiments, this window moving may beaccomplished in a drag and drop-like operation using a human interfacedevice. In a further embodiment, additional instructions—whether of theforegoing types of different types entirely—may be communicated by auser to the client's parent window for additional purposes. As anon-limiting example, the purpose of said additional instructions may beto access and/or control GUI elements such as windows, ribbons, menubars or other menus of said windows. It will be further appreciated thatadditional input instructions may likewise specify configuration and/orsettings-related information relative to any aspect of an embodiment ofthe invention described herein. In a further embodiment, additionalinstructions may likewise be received to resize or otherwise modify thesize of the parent window client.

In a further embodiment still, a scrolling capability may be present toallow one or more users to navigate left, right, up, or down when thesize of the window parent structure exceeds the visible area of alltiles and related GUI contents. It will be readily appreciated that sucha pan and/or scroll feature that would allow one or more users to viewthe contents of a region of the parent window that lie beyond the givendimensions of said parent window may be desired. In a furtherembodiment, display panning and/or scrolling indicators may be presentto provide human users with location and/or size cues. In a furtherembodiment, a function and mechanism to scale necessary GUI componentsto fit within a specified parent window's area may be implemented, asdescribed in the discussion of the window module 260 herein.

In another embodiment, repositioning may include resizing of the windowparent structure within the electronic visual display 208 on which it isshown.

Input instructions 210, 211 received are distinguished for their purposeby the event listener module 215. In an embodiment, the event listenermodule 215 receives said input instructions from one or more human inputdevices connected to an embodiment of the present invention andidentifies said input instructions 210, 211 as being of the userpoint/click 210 type or either of the repositioning 211 type. Onceinstructions are thus identified, the event listener module 150 issuesthe instructions to the event dispatcher module 200 (further describedherein).

In another embodiment, the event listener module may listen for mouseclicks, taps, key presses, gestures, or any other events deemed valid,acceptable, or appropriate either as a user point/click input 100 or asa user repositioning input 125. Events not acceptable or inappropriatefor handling by the event listener module 150 may be sent to a distinctobserver or other event handling system, such as might be provided—in anon-limiting enumeration—by the operating system upon which anembodiment of the present invention executes, to another program, orsimply be ignored.

The person skilled in the art will readily appreciate that input devicesas intended herein need not be limited to keyboards, mice, ortactile/touch screen technologies, but in certain embodiments mayinclude inputs as might be received from sensing equipment. The skilledperson will appreciate that a non-limiting enumeration of such equipmentmight include cameras 401, microphones, or multimodal sensors. Likewise,three-dimensional actions or gestures (such as the wave of the hand or adirector's “cut” gesture), or other sensory inputs, such as might bereceived via voice command using a microphone, through eye control or byway of one or more recognized gestures may be detected using any one ormore sensors or multimodal sensors. In an embodiment of the invention,the event listener module 215 includes a relationship with one or moredevice drivers of human interface devices which communicate with saidembodiment of the present invention. It is contemplated that theimplementation of such user interface component support can beimplemented in such manner as to allow a user click, tap or othersuitable selection specified upon a parent window structure 270 to betransmitted to the appropriate visual element 203. Furthermore, theforegoing dynamic can be implemented in one, two, or more distinctprocesses. In another implementation, a series of out-of-processrenderer components 204 and/or corresponding visual outputs 205potentially having no superjacent transparent window can be configuredto receive and forward user input 210 to a parent process of the windowmodule 260 or the semi-transparent window parent structure 270, or evento an appropriate target visual element 203.

Input instructions received by the event listener module 215 may beintended to ultimately control an aspect of any one or more of themodules of the client 200, various embodiments of which are describedherein, or for other entities auxiliary to it.

Event Dispatcher Module

The input instructions received as events 210, 211 by the event listenermodule 215 are, in an embodiment, issued to the event dispatcher module220. Said dispatcher module 220 processes said input instructions 210,211 to identify the ultimate module and component to which to transmit aspecific instruction. In a series of embodiments, the event dispatchermodule accepts user input 210, 211 for one or more specific positionsthe parent window structure 270, further described herein, andaccordingly dispatches corresponding commands to various modules locatedboth inside and outside the client program 200, including toout-of-process components 204 whose corresponding visual output 205occupies a given tiled 280 area within said window parent structure 270.

Mechanisms or schemes by which to identify the module(s) to which todirect input instructions received by the event listener 215 (e.g.whether to one or more modules within the client program 200 or to oneor more modules auxiliary or external to it) may vary. In an embodimentof the present invention, the event dispatcher module 220 is providedwith information regarding the position of various software objects, GUIelements, and components that occupy the various regions of the windowparent structure module. Such information on relative position withinthe parent window structure or within one or more desktops on which thepresent invention executes may in various embodiments be inquired by theevent dispatcher module 220 and received from the various modules suchas the window module 260 present within said embodiments and furtherdescribed herein, or from the operating system itself.

In an embodiment, the module to which said instructions are intended maybe identified by virtue of the human input device from which saidinstructions are received. This identification method can be simplifiedin embodiments where a human interface device type provides instructionsintended exclusively for a single module within ones of saidembodiments.

In the course of its operation, the event dispatcher module 220 canimplement various synchronization schemes to properly govern itscommunication with various target modules described further herein. Inan embodiment, such communication is necessary to determine the state ofoperation and/or readiness of such a target module to receive theinformation issued by the event dispatcher module 220. This is furtheruseful in any determination and assignment of priority, which in anembodiment, the event dispatcher module 220 may likewise assess anddetermine. Such assessment may be made in light of conditions bothwithin the client program 200 and outside of it, namely within thecomputer system on which said program 200 operates. Such synchronizationand control, it will be appreciated, ensures that multiple instructionsreceived by the event listener module 215 will be routed to the correcttarget module to which each is intended. In addition, it will beappreciated that said routing may further include the participation andinteraction with additional modules not necessarily described herein andnot necessarily a part of the client program 200. Likewise, saidadditional modules may be located external to the client program 200.Both the physical location and the organizational division of theirrespective tasks and operation is contemplated as potentially occurringon any one or more computers. Said operation is likewise contemplated asoccurring irrespective of the relationship between or among of saidcomputers, including without limitation the network topology by whichthey may be interconnected.

In embodiments, conversion of input instructions by the event dispatchermodule 220 into commands intelligible to the various modules for whichthey are intended may be supplemented by a mechanism to manage theissuance of said commands to said various modules. Accordingly, in someembodiments of the present invention, the event dispatcher module 220comprises a synchronization engine whose functionalities includequerying various destination modules present within said embodiments,said modules either expected or expecting to accept input instructions.In certain embodiments, said synchronization engine includes a structureto assign priority and control of the issuance, flow, and routing ofcommands. Accordingly, in certain further embodiments, saidsynchronization engine coordinates with the event dispatcher module 220more broadly to transmit commands to modules directly; in otherembodiments, the issuance of commands is done with the participation ofone or more operating system components. In another embodiment, thecoordination of said commands may be undertaken by various appropriateschemes. Such schemes include, without limitation, implementing acommand queue, synchronizing the issuance of commands with the operationof other modules within an embodiment of the invention, and assigning apriority policy to specific commands or command types. Command issuance,operation, and communication between the event dispatcher module 220 andother modules within an embodiment may be implemented using either asynchronous or asynchronous communication protocol, or a combinationthereof. Likewise, any message passing mechanism, scheme, interface,standard, or protocol may be contemplated for such communication, and beappropriately suited to a specific embodiment. Moreover, any additionalstructure, such as a message or event queue coupled with any othermechanism to assign priority (e.g. when multiple commands are receivedwithin a short interval) and/or control may be likewise contemplated andimplemented for purposes of properly directing input informationreceived from the event listener module 215 to any one or more intendedtarget module.

In embodiments, any one or more modules may in practice receive commandsissued by the event dispatcher module 220. Such modules may include anyone or more of the modules described in various detail herein, likewise,additional modules specific to one or more other embodiments may beconfigured to receive such commands as well. In particular, theout-of-process renderer 204 and non-renderer 202 components modules,further discussed herein, may in some embodiments receive such commandsfrom the event dispatcher module 220 directly. This may particularly bethe case in instances where the command to forward from the dispatcher220 does not require any specific intervention from, or modification orhandling by, the operations setup module 250, further described herein.In another embodiment, commands issued by the event dispatcher module220 may be first sent to the operations setup module 250 and thence toany one or more out-of-process renderer components 204. This may beparticularly desirable or even necessary in embodiments where the setupmodule 250 must interpret commands that it receives from the eventdispatcher module 220 in light of other operational constraints,requirements, or reality. Commands issued from the event dispatchermodule 220 to out of process renderer components 204 include image- orvideo-level configuration instructions to said components 204 thatspecify part or all of the latter's functionality. In anotherembodiment, commands issued by the event dispatcher module 220 mayspecify explicit instructions on assigning one or more out-of-processrenderer components 204 to one or more video feeds 201. It will beappreciated that in embodiments where direct communication between theevent dispatcher module 220 and any one or more out-of-processorcomponents 204 exists, a command structure separate from that discussedherein for the operations setup module 250 may be provided. Accordingly,said separate command structure may be provided within the eventdispatcher module 220 or within one or more out-of-process renderercomponents 204 present to ensure that input received by the eventdispatcher module 220 from the event listener module 215 is properly andultimately converted to intelligible commands for use by theout-of-process renderer components 204. Likewise, in a furtherembodiment, namely one in which a dispatcher module 220, an operationssetup module 250, and one or more out-of-process renderer components 204are present, portions of the command generation process may be sharedamongst all three module types before the out-of-process renderercomponent(s) 204 may in turn respond to said command. It will beappreciated that any appropriate communication protocol and/or messagepassing scheme between the aforementioned modules 220, 204 may becontemplated and implemented, and that no explicit constraint iscontemplated as to the number or variety of connections contemplated inthis regard. Likewise, in embodiments where the operations setup module250 is present, a successive or staged encapsulation mechanism may beenvisioned, in which information issued by the event dispatcher module220 to the operations setup module 250 is interpreted or otherwisefortified by the latter with additional information before beingconverted into a command for the intended out-of-process renderercomponent 204 and eventually issued to the latter.

While significant discussion thus far has been directed towardcommunication between the event dispatcher module 220 and the operationssetup module 250 and out-of-process renderer components 204, it will bereadily apparent that input information received by the event dispatchermodule 220 must likewise be transferred as commands to various other keymodules, including, non-limitingly, to such modules as the non-renderercomponents 202, visual elements 203, arraying module 203, all of whoseoperations will be further described herein.

In particular, the issuing of instructions directly to non-renderercomponents 202 and to visual elements 203 (or to the latter by way ofnon-renderer components 202) may be further appreciated as beingimportant in driving user interaction of the client program 200. This isnamely due to the nature and role of the visual elements 203 as being aprincipal point of contact and control between users and embodiments ofthe client program 200. Individual discussion of, as well as therelationships between visual elements 203 and non-renderer components202 is provided in subsequent sections herein. Irrespective of whetherthe event dispatcher module 220 accepts input 210, 211 for which it 220provides immediate conversion to a single or complete command, it willlikewise be appreciated that in an embodiment, the setup module 250 orthe window module 260 may likewise receive output from the eventdispatcher module. While the latter two modules will be discussed ingreater detail herein, their reception, whether of instructions orcommands from the event dispatcher module 220, is important in theconversion of user input in important interface-related tasks such asissuing changing the dimensions of said the window parent structure inwhich embodiments run.

In another embodiment, the event dispatcher module 220 may be partiallyimplemented within one or more visual elements 203, further discussedherein. This may be desirable, for example, in cases where a visualelement 203 is a user interface control configured to receive to receiveuser input 210, 211 to control the operation of the client program 200.Alternatively, the event dispatcher module 220 may conversely issuecommands to a non-renderer component 202, which in turn directs acorresponding visual element 203, thus indirectly controlling visualelement 203 from the dispatcher module 220. A non-limiting example ofthe latter operation may occur in cases where the visual element 203 isa digital on-screen graphic (such as a logo, information mark, or iconappearing at a corner of a tile) or a motion graphic (including a 3D orother animation to be superposed or overlaid—as further discussedherein—atop the visual output 205 resulting from the work of one or moreout-of-process renderer components 204).

Additionally, in an embodiment, the event dispatcher module 220 acceptsinput 210, 211 that it 220 interprets into instructions or commands fordirecting the operation of the arraying module 230. In a non-limitingenumeration, such commands may include user instructions for theintended positioning, layout, and configuration within the window module260 of the various feeds 201 received.

It will be further appreciated that the interpretation or conversion ofuser input 210 to instructions or commands may be done entirely by theevent dispatcher module 220 as discussed for various embodiments herein,or be complemented to any degree by interaction with other components,including modules of operating systems upon which embodiments execute.

In a further embodiment, the event dispatcher module 220 furthercomprises one or more suitable control structures (such as a semaphoreor a priority queue) to ensure that the various commands that itconverts and dispatches to other modules are issued in accordance withan appropriate priority policy.

OOP Software Renderers and Visual Outputs

As discussed herein, embodiments receive video feeds 201 from cameras401 or other signal sources outside the client program 200. Embodimentstypically receive feeds which must be decoded and rendered to beintelligible to be used by embodiments and to convey information usefulfor human users. In a manner analogous to the paradigm described hereinfor non-renderer components 202 and visual elements 203, namely whereinthe latter constitutes the visual result (or realized output) of theformer, the out-of-process renderer components 204 perform the renderingand decoding tasks upon received video feeds, while the visual outputs205 correspond to the video feeds' 201 stream once decoded and rendered.

Consistent with its name, an out of process renderer component 204 is,in embodiments, located and made to execute in a process external to theone corresponding to the client program 200. It will be appreciated thatsuch a configuration mitigates the serious performance pitfalls likelyto occur when a sizable number of feeds would simultaneously requirerendering, potentially slowing down the performance of said clientprogram 200.

The operations setup module 250 instantiates an out-of-process renderercomponent 204 upon receiving a command to do so by the event dispatchermodule 220. Such a command is typically issued by the latter 220 furtherto the receipt of an event 215 in the form of user input 210, 211 tothis effect, as a GUI-based selection or other issued command by a humanuser within a part of the client program 200. In the course ofinitialization, the operations setup module 250 matches the video feed201 requiring rendering with an out of processes renderer component 204that it selects (using any appropriate decision-making scheme) fordecoding and rendering said feed 201. The out of process renderercomponent 204 carries out its operation in accordance with anyadditional instruction provided to it (including, without limitation,performance settings and/or other execution parameters) provided by theoperations setup module 250, also consistent with user input receivedinitially 210, 211, or with any previously-specified policy.

The output of the renderer component 204 is referred to as a visualoutput 205. In embodiments, the visual output 205 typically correspondsto the image data received from the video feed 201 converted to videostream format and which may be viewed as such on a computer. Asmentioned herein, this visual output 205 is subsequently issued to thecomposition manager 206 for compositing with any one or morecorresponding visual elements 203 intended to be superjacently alignedwith said visual output 205 (FIG. 4a , FIG. 6, FIG. 7).

In another embodiment, the arraying module 230 may issue instructions toan out-of-process renderer component 204, to its visual output 205,and/or to the composition manager 206 instructions on positioning withinthe window parent structure 270 and/or z-order hierarchy of the visualoutput 205 and visual elements 203 to cause the visual elements 203result u (discussed herein) to be displayed subjacently to thetransparent tiles 280 occupying the semi-transparent window parentstructure 270 but superjacently to said visual outputs 205 (FIG. 4a ).In another embodiment, direct communication channels (i.e. withoutintermediary module) may be implemented between visual elements 203 andout-of-process renderer components 204 so that the latter may receiveinstructions from the former on positioning and relative z-orderhierarchy to be subsequently applied. It will be appreciated that aplurality of visual elements 203 superjacent to the visual output 205can be composed in accordance with a specific order of priority. Theforegoing order of priority can in turn determine the z-order that thevarious visual elements 203 occupy. Such an order of priority can bedetermined by a pre-specified sequence, or be specified by a human userin the course of operation of an implementation. Nonetheless, the resultof such specification can be the establishment of a z-order hierarchy ofcorresponding visual elements 203. As a result of such a hierarchy,multiple non-renderer components 202 can be combined, and respectivelycomposed above a visual output 205. In implementations, such visualelements 203 and/or non-renderer components 202 can even reside withinthe client program 200.

In view of the foregoing, it will be appreciated that thesemi-transparent window parent structure 270 may be conceived of as bothan abstract construct and tangible layer whose purpose is to occupy thehighest z-order of any component described herein. As such, user input210, 211—particularly of a pointing-, tapping-, clicking-, ordragging-like nature, will be first received by the event listenermodule 215 and subsequently routed to the intended GUI element to beaccordingly handled by an underlying software (or hardware) component.

In a manner analogous to the cases further described herein for thenon-renderer components 202, the out-of-process renderers' 204 operation(with the appropriate substitutions) is monitored by the watchdog module240 for anomalous operation. Here again, anomalous operation is to besimilarly understood as a widely encompassing concept and term that mayrefer to any incomplete function, non-function, or performance-relatedissue. Accordingly, the watchdog module 240 may monitor the executionstatus of any one or more out-of-process renderer components 204 andfollowing detection of any anomalous operation, restart saidcomponent(s) 204. Likewise, the watchdog module 240 may terminate anyout-of-process renderer component 204 following detection of anyanomalous operation of said component(s) 204.

In a manner analogous to the parameter set described for thenon-renderer components 202, and as further discussed herein, thewatchdog module 240 may be configured to receive a set of any number ofcondition parameters to either assert with certainly or alternativelydeduce instances in which anomalous of one or more out-of-processrenderer components 204 may occur. Upon occurrence of said instances,whether deduced or certain, the watchdog module 240 may inform theoperations setup module 250 to issue a command to the window module 260directing the latter to prompt a human operator with an exceptionmessage and the option to restart any one or more anomalouslyfunctioning out-of-process renderer component(s) 204. Upon reading saidmessage and prompt(s), a human user may be presented with a “yes” optionto restart or “no” option to avoid restarting any one or moreout-of-process renderer components 204 having operated anomalously. Inanother embodiment, a similar set of “yes”/“no” prompts may likewise beoffered, but with the option to terminate any one or more out-of-processrenderer components 204 having operated anomalously.

It will be appreciated that the execution of out-of-process renderercomponents 204 outside the client program 200 provides embodiments withan important performance advantage, particularly as the number of videofeeds to decode and/or render increases. The execution of suchcomponents 204 outside obviates any wait, lag, or delay that would beintroduced if rendering were done for an equally large number of feeds201 within the execution thread of said client program 200. A largenumber of high-resolution streams may thus be rendered, including with aframerate superior than would be the case if the components' 204execution were integrated within the client program 200. Similarly, thisexternal thread execution strategy encourages data encapsulation whilefacilitating the fault isolation component described herein as itshields the client program 200 from undesired effects arising from thetermination, restarting, and/or any other corrective or contingentaction to be taken as a result of anomalous operation of any one or moreout-of-process renderer component.

Window Module

The window module 260 plays an important part in embodiments in its dualrole as the recipient of user input 210, 211 as well as the vehiclethrough which the composited and arrayed result of video feed andnon-video feed components is visually laid out for human users. Thewindow module 260 receives events from embodiments of the event listenermodule 215 which it 260 accordingly uses to direct the behavior of thewindow parent structure. Likewise, FIG. 3 and FIG. 4a togetherillustrate the relationships between the window module 260 and thevarious components, elements, and modules with which it 260 interacts.

Embodiments may run on computer operating systems configured to supportwindowing functionality. Accordingly, said windowing functionality maybe implemented or configured to support portions of the window, icon,menu, and pointer paradigm known in the art. In further embodiments, thewindow module 260 implements a main window of the client program 200 inwhich the principal content to be displayed is laid out, consistent withinstructions received from the arraying module 230, further describedherein. In embodiments, the window module 260 may also provide scalingfunctionality so that various interface components may be displayed atdifferent pixel densities. The window module 260 may likewise provideadditional conversion capabilities as to allow various GUI features andvisual elements to be displayed using the traditional coordinates anddirectionality expected in right-to-left (RTL) languages as shown inFIG. 4 b.

In its framework role in many embodiments, the window module isconfigured to implement a semi-transparent window parent structure 270.The window parent structure 270 occupies an important part of thedisplay 208 on which embodiments run, and furthermore comprises anynumber of standard window GUI elements, such as menus, ribbons, andbuttons with which a human user may interact, in addition to text andnotification areas. A sketch showing the contextual relationship of theforegoing parent structure 270 with elements to which it relates will bediscussed presently and is shown in FIG. 5a . Accordingly, a screenshotdepicting a possible layout for the foregoing is shown in FIG. 5 b.

An important feature of the window parent structure 270 is the presenceof an array of transparent tiles 280, whose nature will be describedshortly. The number of transparent tiles 280 may vary among embodiments;alternatively, it may be a settable parameter within a given embodiment.In the latter case, the tile count parameter may be specified 210, 211by a user to the event listener module 215, which accordingly dispatchesthe input in accordance with said parameter setting to the eventdispatcher module 220, ultimately directing the arraying module 230 tolay out said number of transparent tiles in preparation for content. Thespecific disposition of said tiles may be hardcoded or alternativelyspecified by way of a further parameter or group of parameters,preferably contained and managed within the arraying module 230,although in another embodiment, such parameter settings on tiledisposition may be contained or otherwise managed in any one or moremodules deemed appropriate. The window module 260 is said to implement asemi-transparent window parent structure 270 over at least a portion ofthe total window area that it 260 covers. The notion ofsemi-transparency here may be understood to accurate for either of tworeasons; namely, because a portion of said structure 270 is transparentand another portion not, or because the entirety of said structure ispartially transparent, allowing (potentially multiple) content layersfor transparent tiles 280 to be defined and with which to be interacted.In an embodiment, the semi-transparent window parent structure mayoccupy a rectangular region within the output of the window module 260.In a further embodiment, the precise area of the parent structure may bespecifiable and/or modifiable parameter in a manner similar to thatdiscussed for the number and disposition of tiles 280. Although thewindow parent structure 270 may be thought of canonically as arectangular area defining the client program 200, in a still furtherembodiment, no specific constraint on the shape of said parent structure270 need exist.

In this context, a transparent tile 280 is a delimited constituentportion of a semi-transparent window parent structure 270. Said tile 280is likewise said to define a wholly or partially transparent regioncomprising the composited contents of a visual output 205 and thelatter's attendant visual elements 203 disposed in accordance with aspecific logical superposition ordering or hierarchy. In a mannersimilar to the window parent structure 270 described previously, thetransparent tile 280 may be thought of canonically as a rectangular areawithin the window parent structure 270; accordingly, in anotherembodiment, no specific constraint on the shape of said parent structure270 need exist. In yet another embodiment, no constraint on the numberof visual elements 203 and visual outputs 205 need exist.

A tile 280 may be entirely transparent, or alternatively specify opaqueregions, such as to accommodate one or more visual elements 203, such asa clock displaying the time, elapsed playing time, borders, or otherinterface controls as further described herein. A specific z-ordering oroverlapping hierarchy is generally enforced, in which various visualelements 203 and visual outputs 205 are typically made to line up withspecific (x,y) positions behind or below a tile. In embodiments, suchalignment may be accomplished by way of a command issued to any one ormore of the operating system on which the client program 200 runs, thenon-renderer components 202, or the out-of-process renderer components204, or alternatively by the combined efforts of the composition manager206 and the arraying module 230. In another embodiment, said alignmentmay be partially implemented by way of interaction with another module.

It will be appreciated that functionality may be implemented by which toscale necessary visual elements 203 to fit within a tile 208 so thatsaid tile 208 may in turn adequately fit within a specified windowparent's 270 area. Likewise, the window parent structure may in anotherembodiment be configured to variously accept user input 210, 211 tochange its dimensions or be otherwise resized, with a series ofparameters stored or being set within any one or more of the arrayingmodule 230, window module 260, or any other location. In anotherembodiment, resizing of either transparent tiles 280 and/or thesemi-transparent window parent structure 270 within a display 208 mayalso be managed dynamically, such as when the window parent 270 isexpanded to fill the entirety of a display 208. Likewise, the windowmodule 260 can accept user instructions, such as might be received fromthe event listener module 215, to allow the window parent structure 270itself to be resized, whether said resizing operates to cause saidwindow parent 270 to fill the entire display 208 on which said parent270 is output, be returned to a former dimension, or be provided anentirely different set of dimensions. Said resizing is implemented byany combination of the arraying module 230, further discussed herein,operating in tandem with window handling functionality provided by theoperating system. It will be appreciated that the degree to which saidresizing of the window module 260 output and specifically the windowparent structure 270 is provided by either the arraying module 230itself or the operating system, or to any specific proportion of thelatter two entities, may be subject to variation.

It will be appreciated that in an embodiment, the semi-transparentwindow parent 270 may be a window, or alternatively incorporated withina window. It will be further appreciated that in said embodiment, a usermay resize said semi-transparent window parent 270 by resizing thewindow to which it 270 corresponds, or in which it is contained. It willbe still further appreciated that said window may additionally includewidgets, ribbons, and menus. User input 210, 211 is likewise received ator by said parent window 270.

In another embodiment, a change in size of the window parent 270 may becoordinated in such manner as to accordingly and proportionally changethe dimensions of any and all tiles 280 that it contains. Thus, therelative aspect ratios of both the window parent 270 and each of thetiles 280 may be maintained irrespective of any resizing operations. Ina further embodiment, the visual output 205 content within each tile 280may likewise be scaled in proportion with the change in dimensions ofeach tile 280, likewise in relation to the change in dimension of thewindow parent 270. In embodiments in which the visual output 205 doesnot scale in proportion with the change in dimensions of itscorresponding tile 280, additional interface elements, such as pan andscroll bars may be provided to assist and guide human users. It will beappreciated that said additional interface elements need not beconstrained or be otherwise specific to a single area, such as abouteach tile 280, on or near the window parent structure 270, on any partof the window module 260 output to the display, or even external to theclient program.

With reference to FIG. 6, human user input 210 may be received for aspecific position within a window parent 270 structure; in a furtherembodiment, said input 210 may relatedly be intended to actuate any oneor more visual elements 203 within a specific tile 280. In anembodiment, said input 210 is received for a specific (x,y) coordinatesof the semi-transparent window structure 270, with said coordinatesbeing received by any combination of the event listener module 215 andthe operating system's window handling system, converted into a command,and ultimately issued to the intended non-renderer component 202 and/orout-of-process renderer component 204, as required.

In another embodiment, the watchdog module 240, further describedherein, monitors the operation of the out-of-process renderer components204. In a still further embodiment, the watchdog module 240 performssimilar monitoring of the non-renderer components 202. Upon saidwatchdog module's 240 detection of anomalous operation of either type ofcomponent 202, 204, the window module notifies the operations setupmodule 250 (further described herein) by way of any appropriate messagepassing mechanism. In turn, the operations setup module 250 accordinglyissues a notification signal to this effect to the window module 260,which accordingly issues a specific exception notification message to bepresented to the human user. In an embodiment, said notification messageis presented on the display 208. It will be appreciated that the windowmodule's 260 presentation of said notification message need not beconstrained to any single type, wording, or form, and may provide anylevel of detail, such as which component operated anomalously, and theseverity thereof being non-limiting examples. Likewise, it will beappreciated that the aforementioned notification issued by the windowmodule 260 may vary to include any one or more of a popup message window(whether separate or incorporated within said semi-transparent windowparent structure 270), a change in color to any one or more interfaceelements, or any other visual indicator. In a further embodiment, anon-visual indicator may accompany the latter visual notification, or beindependent therefrom. Said notifications may likewise be issuedremotely, such as through a network and which may be delivered to one ormore recipients, in a non-limiting enumeration, via email, text message,or other alert. In another embodiment, said message may likewise providea series of options with which to proceed, such as whether to terminateor restart the anomalous component(s), or to proceed by taking anyadditional measures.

The hierarchical implementation of a semi-transparent window parentstructure 270, the latter receiving the highest z-order ranking isimparted, forms the basis of the desired performance enhancementsdelivered by embodiments discussed herein. The next-highest z-order isoccupied by visual elements 203 and the next highest by the visualoutputs 205 of out-of-process renderer components 204. This hierarchicallayering, which combines especially the out-of-process operation ofrenderer components 204, in addition to the composition capabilitiesresulting from the work of the composition manager 206 (afforded, inembodiments, to varying degrees by the operating system), results insignificant performance enhancements. Such performance enhancements arepossible because the structural interconnections disclosed hereinbetween the semi-transparent window structure 270, composition manager206, and out-of-process renderer 204 and non-renderer 202 componentsimplement a low-resource client program 200 while maximizing thecapabilities of the most processing-intensive components of the system.Doing so alleviates many performance-related shortcomings, whileminimizing the fault isolation shortcomings which are clearlyidentifiable (and options for corrective action) presented to users.

Arraying Module

The arraying module 230 receives commands from the event dispatchermodule 220, visual elements 203, and composition manager 206 that ituses to configure, position, and dimension visual outputs 205 resultingfrom any one or more out-of-process renderer components 204instantiated, initialized, and made operational within the clientprogram 200. The composition manager 206 composites visual elements 203of non-renderer components 202 with the visual outputs 205. The lattervisual outputs 205 typically correspond to scenes 402 converted intovideo feeds 201 by capture devices 401, and are the processing output offrom out-of-process renderer components 204. The arraying module 230configures outputs received from the composition manager 206, typicallyas the combined integral content (consisting of visual output 205 andall visual elements 203) to be forwarded to the window module 260 usingany communication protocol and displayed within a space corresponding tothe dimensions of a single tile 280 occupying a portion within thesemi-transparent window parent structure 270.

It will be appreciated that various interactions involving the arrayingmodule 230 are described in the sections detailing the function ofvarious other modules herein; to avoid repeating them all in thissection, the reader may consult said sections for additional details onthis module 230.

In embodiments, the arraying module 230 forwards commands to the windowmodule 260 specifying the positioning of various visual elements 203corresponding to multiple tiles 280 to occupy a z-order placing saidvisual elements 203, when appropriate, atop or otherwise superjacent tothe visual outputs 205 to be specified to occupy any one or more tiles.In another embodiment, such forwarded commands may be subject totemporary delay as portions of tasks intended for prior completion bythe composition manager 206 further described herein are carried out bythe latter. In some embodiments, said tasks, and related forwardedcommands, may be forwarded to the operating system upon which the clientprogram 200 executes. This may be necessary in cases where thecomposition manager 206 functionality is provided in whole or in part bythe operating system.

Information on which tile 280 slot within the window parent structure270 need be occupied by the output of the composition manager 206(typically a combination comprising a visual output 205, in variousembodiments disposed subjacent to any one or more visual elements 203).It will be appreciated that for those tiles 280 on which no visualelement 203 has been specified (or need be present), the arraying module230 issues positioning instructions to the window module 260 on whichtile 280 slot within the semi-transparent window parent structure 270the corresponding visual output 205 alone need be placed. Much morerarely, visual elements 203 to be disposed within one or more tiles 280(but without a corresponding visual output 205) may in anotherembodiment be issued to the composition manager 206 and then to thearraying module 230 for disposition within the parent structure 270.

In embodiments, the arraying module is configured to not only impart arelative position to the individual outputs resulting from thecomposition manager 206 by forwarding positioning information to thewindow module 260, but to also cooperate with the latter modules 206,260 to enforce relative z-order hierarchy (i.e. superjacency/subjacency)of the content resulting 203, 205 from the various renderer 204 andnon-renderer components 202 specified, instantiated, and initialized foreach of various tiles in various use cases of the client program 200. Infurther embodiments, the arraying module 230 issues instructions to thewindow module—a portion or all of whose positioning duties may beimplemented by the operating system—to display visual elements 203and/or visual outputs 205 subjacently. In still further embodiments,instructions to display visual elements 203 superjacently to said visualoutputs 205 may be further issued by the arraying module 230 to thewindowing module 260 for this purpose.

In another embodiment, the arraying module may receive user input 210,211 from the event listener module 215 by way of the event dispatchermodule 220 for changing the dimensions of said semi-transparent windowparent structure, and/or any window resulting from window module 260output. In a further embodiment, said instructions may be received andfor causing the semi-transparent window parent structure 270 to bere-dimensioned in accordance with received user input 210, 211, whichmay non-limitingly consist of a drag-to-expand, drag-to-shrink, ortextual specification of dimensions by a user.

In a still further embodiment, and as further described herein, thearraying module 230 may change the dimensions of a window parent 270,with the tiles 280 contained the latter 270 automatically changing inproportion with the change in dimension. In a yet further embodiment,the scaling of visual element 203 and/or visual output 205 content inone or more tiles 280 within the window parent 270 may likewise bescaled to the new dimensions specified.

Composition Manager

As disclosed herein, the composition manager 206 is a component—andtypically functionality offered and/or managed by the operatingsystem—to perform window compositing. In another embodiment, acomposition manager 206 may likewise include processing functionalitysuch as scaling, rotation, and warping. In a still further embodiment,each visual output 205 or tile 208 may have a composition manager of itsown. In yet a further embodiment, composition may be managed by one ormore operating system components. Thus, a composition manager 206 may beappreciated as a sort of abstraction for any of severally locatedsystem- or operating system-level services to perform the aforementionedcompositing and other functionalities.

It will be appreciated that in various contexts, composition as a termmay refer to the related notions of establishing the value, color, andintensity of pixels as a function of the various logical structures saidpixels collectively depict. Relatedly, the term may also refer to thestacking or superposing of various controls atop one another. Inembodiments contemplated herein, the notion of composition should beappreciated as a mixed-mode mixture of immediate and retained modecomposition.

As disclosed in various sections herein, the composition manager 206receives as inputs any one or more visual outputs 205 and/or any one ormore visual elements 203 and respectively composites them, typicallysuperposing the latter atop the former, to visually create the illusionthat both are truly aspect of the same scene. The composition manager206 then issues the composited tile content as it should appear at itsintended tile 280 slot within the semi-transparent window parent 270once displayed 208.

Certain operating systems allow each process to perform its owncomposition tasks within the framework of said operating system.Increasingly, the operating system allows client applications to accessthe hardware capabilities of graphics cards present within a computer toperform such composition duties. Thus, the client program 200 need notnecessarily rely on computer processor cycles to perform thecomposition; in combination with out-of-process thread execution notionsdiscussed herein, such offloading to specialized hardware allowssignificantly improved performance targets to be met. Such performanceimprovements are valuable in light of the higher resolution images (e.g.4k) becoming increasingly commonplace, as they do not slow down theclient program nor limit the number of video feeds 201 to be rendered204 to merely the number possible.

It will be appreciated that the composited 206 tiles 280 disposed withinthe semi-transparent window parent structure are intended to provide therelative appearance and operation of said components into a singleseemingly integrated GUI experience as seen from the perspective of ahuman user.

Software Non-Rendering Components and Visual Elements

Software non-renderer components 202 play an important role inembodiments described herein. While renderer components 204 ensure thatvideo feeds 201 are converted into equivalent visual outputs 205 forcompositing 206, arraying, and ultimately for presentation by the windowmodule 260, in embodiments, non-renderer components 202 and theirassociated visual elements 203 implement software controls thattypically provide human users with components with which said users mayinteract, typically by supplying user input 210, 211 as discussed herein(FIG. 8). Alternatively, non-renderer components 202 may have as theirrespective visual elements 203 far less interactive components. Usersmay accordingly interact less or not at all with the latter variants.

A typical example of non-renderer components 202 as contemplated forembodiments and which may be considered to be in the interactivecategory non-limitingly include ActiveX components. Such components 202provide for the creation, as their corresponding visual elements 203, ofsoftware objects which users may interact, typically by way of a userinput 210, 211 further discussed herein. Such software objects mayinclude pictographic visual elements (e.g. resembling media controls301, video progress bars 302) and implement corresponding functionality.In other cases, A set of visual elements 205 an ActiveX control (i.e.non-renderer component 202) that implements a more comprehensivefunctionality—such as an entire player—may for example be implemented.

Non-renderer components 202 considered to be in the less interactivecategory may likewise be contemplated for embodiments. These components202, as deployed within one or more transparent tiles 280, are referredto as visual elements 203 of the same 280. Examples of visual elementsmay non-limitingly include digital on-screen graphics, such as a logo,icon 304, or information mark 303, appearing within a part of or at acorner of a tile 280. Other examples of visual elements non-limitinglyinclude motion graphics, such as an animated variant of an on-screengraphic, including a 3D image, textual animation, or a moving annotationsuch as contour tracking of a specific target. Where functionallyappropriate, non-renderer components 202 may further interact with or besupplied with data by one or more external sources, such as an imageprocessing application.

In a manner akin to the case of out-of-process renderer components 204,further described herein and whose execution is desired to occur outsidethe process of the client program 200, non-renderer components 202 andtheir associated visual elements 203 are likewise optimally desired tooperate out of process for similar reasons (i.e. to obtain similarbenefits and avoid similar disadvantages).

In another embodiment, renderer components 202 may execute within thesame process as the client program 200. In a further series ofembodiments, the corresponding visual elements 203 may likewise bespecified to run in any one of the same process as the client program,the same process as one or more out-of-process renderer components, oran entirely separate process altogether. It will be appreciated that theexecution process within which one or more non-renderer components 202may execute may be determined based on the specific nature, function,and/or operation of the non-renderer component 202 and/or itscorresponding visual element 203 itself.

It will be appreciated that the degree to which each of these visualelements 203 (and their corresponding non-renderer components 202)specified and instantiated for a tile 280 may interact and cooperatewith out-of-process renderer components' 204 respective visual outputs205 varies in accordance with the specific nature of the non-renderercomponent 202.

Visual elements 203 of non-renderer components 202 are typicallysuperposed atop the visual outputs 205 of out-of-process renderercomponents by the composition manager 206, further described herein. Foran individual tile 280, a user may specify (for example, from aselectable list or selection menu accessible to the client program 200,or by any other appropriate means) the non-renderer components 202 toinstantiate for a specific tile 280. Such selection may be made via anyuser input 210, 211, with the event listener module 215 issuinginstructions to the event dispatcher module 220 to instantiate saidnon-renderer component 202. Said non-renderer component 202 theninitializes any visual element 203 required for said non-renderercomponent 203 to operate within a tile 280. In an embodiment, the visualelement 203 of said component 202 may subsequently communicate with theevent dispatcher module 220, which in turn issues notification of theinstantiation of said component to the operations setup module 250 tologically couple the newly instantiated no-renderer component 202 withthe specific out-of-process render component 204 with which it isintended to cooperate. Thus, a collection of visual elements 203corresponding to one or more non-renderer components 202 for eachindividual tile 280 (FIG. 9) may be constructed for cooperation withvisual outputs 205 corresponding to rendered 204 variants of feeds.Furthermore, the event dispatcher module 220 may, upon receipt ofinstructions, typically by way of user input 210, 211 (from the eventlistener module 215 or a subportion thereof local to one or more visualelements 203), may cause the event dispatcher module 220 to executecommands consistent with the operation of said non-renderer component202 by issuing instructions, for example to the arraying module 230 orto the operations setup module 250. Instructions issued to the arrayingmodule, may comprise information on specific coordinates within thebounds of a specific tile 280 (or alternatively, within thesemi-transparent window structure 270 or any part of any output from thewindow module 260) at which the visual elements 203 should be disposed.Instructions issued to the operations setup module 250, as describedherein, may include information or commands to associate said visualelements 203 with specified non-renderer components, or ensure that noexception or incompatibility between said non-renderer component 202 andout-of-process renderer component 204 intended for a specific tile 280might exist or otherwise occur. In another embodiment, visual elements203 corresponding to a non-renderer component 202 and intended forsuperposition atop the foreground video rendered 204 by a visual output205 may be shared with the composition manager 206, which carries outcompositing tasks to amalgamate all visual elements 203 intended forcomposition with a visual output 205 for purposes of producing a tile280 to be issued to the arraying module 230 for later inclusion to thewindow module 260 and ultimate display 208, further described herein.

In another embodiment (FIG. 10a , FIG. 10b ), non-renderer components202 for a series of tiles may be specified a priori for subsequentcomposition 206, further described herein.

Visual elements 203 may be appreciated superposed or overlaid—as furtherdiscussed herein—atop the visual output 205 resulting from the work ofone or more out-of-process renderer components 204).

It will be appreciated that just as the presence and disposition ofnon-renderer components 202 may be either specified in bulk (FIG. 10a )within a number of tiles 280 of the semi-transparent window structure270 or alternatively specified and disposed by a user on a tile-by-tilebasis (FIG. 9), said non-renderer components 202 may be combined, evenwithin a single tile 280, each component operating in-process orout-of-process. No limit other than practical considerations or thetechnical constraints of a non-renderer 202 component itself need beplaced on the execution of a component 202.

While the work of positioning visual elements 203 is the task of thearraying module 230, further described herein, it will be appreciatedthat the non-renderer components' 202 visual elements 203′ are inembodiments typically situated superjacent to the visual outputs 205received from the latter's respective out-of-process renderer components204. Recall that the out-of-process renderer components 204 typicallyrender the scenes 402 captured 401 by one or more video feeds 201received by embodiments. This allows for a typical tile 280 layout inwhich the scene content 402 corresponding to a video feed 201 is in thebackground, with specified visual elements 203 for each of said tiles280 superposed and occupying the foreground (FIG. 5b , FIG. 6, FIG. 7).

It will be appreciated that the detection of anomalous operation of anyone or more non-renderer components 202 can in embodiments beimplemented in a manner similar to that discussed herein forout-of-process renderer components 204, namely by way of communicationbetween the watchdog module 240 and each non-renderer component 202.Likewise, a condition parameter set similar to the one described hereinfor detecting the anomalous operation for out-of process rendercomponents 204 may be contemplated, with the appropriate modifications,to accordingly detect the occurrence of an one or more non-renderercomponents' 202 anomalous operation.

It will likewise be appreciated that detection of anomalous operation ofany one or more non-renderer components 202 can in embodiments beimplemented by way of a two-way communication channel between the eventdispatcher module 220 and each one of said non-renderer components 202,or alternatively communicated by the respective visual element(s) 203resulting from the execution of said component(s) 202 to the eventdispatcher module 220. The discussion provided regarding a set ofcondition parameters to detect the occurrence of specific operationalissues may likewise be applied for such embodiments.

Anomalous operation of non-renderer components 202 may be terminated,restarted, and handled in a manner akin to that described forout-of-process renderer components 204. For example, options to proceedfollowing anomalous termination of one or more non-renderer components202 may include a prompt or popup window that provides one or more userswith a positive or negative decision to terminate or restart said one ormore component(s) 202.

Scenes and Cameras

As discussed in various sections herein, embodiments may (FIG. 11)capture scenes 402 of any type, sort, or nature in any conceivableenvironment. Said scenes 402 may be captured using a camera 401 or anyother appropriate capture device, which converts said scenes 402 intoone or more video feed 201. The format of said feed 201 is not intendedto be constrained to any format but may be of any format, variety, ortype, and follow any technical specification. Variants on splitting,subtracting, adding, and/or merging content into feeds 201 may also becontemplated. To cite several non-limiting examples, such variants mayinclude removing color information from a feed as collected from a(color) camera 401, adding colorizing information, or applying an imageregistration algorithm to one or more scenes 402 captured by one or morecameras 401 to stitch the foregoing into an enhanced feed 201 covering awider scene 402 area than the original constituent captures.

Video Feeds

Video feeds 201 are captured using any one or more capturedevices—namely cameras. However, such feeds 201 need not be receivedfrom video cameras but may be provided by any other one or more signalsources. In embodiments, such video signal capture devices may bedisposed without limitation about any one or more geographical ortopographical zones, areas, locations, or territories, and cover scenes402 of any conceivable variety, such as a country's highway system, abusy shopping mall, a remote desert site, or a lunar mission. Inaddition, the coverage area captured by said capture devices may beentirely unconstrained, and may be variable and/or uncontiguous.Likewise, the nature of the image and/or video data acquired by saidcapture devices may differ from one said capture device to anothercapture device disposed within an embodiment. Such differences mayinclude, without limitation, the actual scene(s) 402 to capture and theformat of the images produced or output by said capture devices 401,including the resolution, framerate, and data encapsulation type of saidimages to capture and share. In another embodiment, additionalinformation may be captured by said capture devices, such as sound ordepth. In a still further embodiment, such additional information tocapture may include non-visible parts of the electromagnetic spectrum,such as the infrared region. Furthermore, video feeds 201 captured andprovided to an embodiment may be accessible exclusively by saidembodiment, or may alternatively be provided and/or shared in accordancewith a specific access control policy. Feeds may likewise be capturedand shared with embodiment live—whether immediate or with negligible orminimal delay. In another embodiment, the feeds may be retrieved byembodiments from a location in which they may have been previouslystored; such playback be deferred or otherwise time-shifted.

Operations Setup Module

The operations setup module 250 performs several managing roles withinembodiments. It may receive the video feeds 201, typically from camerasand/or other capture sources described herein, said feeds beingcollected external to the client program 200. Furthermore, theoperations setup module 250 receives instructions received from theevent dispatcher module 220 to issue specific configuration-relatedcommands to the window module 260 as further discussed herein, and fromwhich it also receives operational feedback relating to the condition ofwindow module 260 output, including without limitation, thesemi-transparent windows parent structure 270 and the individual tiles280 and the contents therein, shown on the display 208. While inembodiments, the operations setup module can 250 receive the feed andpass it through to 204, it may likewise be possible for the operationssetup module 250 to pass an instruction to the out-of-process renderercomponent 204 so that the latter may be responsible for receiving videofeeds 201 directly. Such direct routing of feeds 201 to out-of-processrenderer components 204 may be configured to advantageously reducetransmission overhead.

The setup module 250 also participates in message passing operations,particularly between event dispatcher module 220 and window module 260and watchdog module 240 and window module 260, as further discussedherein.

Likewise, the operations setup module 250 communicates with a resourcepool comprising a number of renderer components 204 whose role andinterconnections are further described herein. For such interaction, thesetup module 250 is typically provided with the ability to accesscomponents outside the client program 200. Upon receiving notificationfrom the event dispatcher module 220 that a new video feed 201 is to beadded to a tile 280 within the semi-transparent window structure 270,the setup module 250 typically instantiates one or more out-of-processrenderer components 204 to which it 250 has access, and furtherinitializes said out-of-process renderer component(s) 204 for operationwith an embodiment. Furthermore, initialization typically involves theallocation of an available out-of-process renderer component 204 forsubsequent association with a specified video feed 201.

When an out-of-process renderer component 204 is available, the setupmodule 250 initializes it. In another embodiment, a specific policy maybe elaborated or live selection may be made among any availableout-of-process renderer component 204 available to the setup module 250.In cases where the watchdog module 240, as further described herein,detects anomalous operation of a specific out-of-process renderercomponent 204, the operations setup module 250 restarts or terminatessaid component 204, in accordance with any specific policy, instruction,input 210, 211, or other decision supplied to it by the watchdog module240 and/or the event dispatcher module 220.

In an embodiment, the operations setup module 250 may be furtherconfigured to communicate the presence or other availability-relatedstatus of any one or more out-of-process renderer components 204. In afurther embodiment, should a specified or otherwise desiredout-of-process renderer component 204 be unavailable at a given time,the operations setup module 250 is configured to share the status withthe window module 260, which can in turn issue a user-intelligiblemessage to the display 208. Likewise, the setup module 250 may share asimilar transaction with the window module when no component 204 isavailable. In such cases, one or more additional information messagesmay be issued to a user. In a still further embodiment, the user maylikewise be provided with the option to specify, via any appropriatecombination of input 210, 211, whether to deallocate or free a renderercomponent 204 currently in use for any specific video feed 201 andoccupying a specific tile 280 within the display 208.

In addition, the operations setup module 250 also communicates with thewatchdog module 240, in a cooperative capacity to implement messagepassing duties from said watchdog module 240 to the window module 260 asfurther elaborated herein.

Watchdog Module

The watchdog module 240 is an important component involved in theoverall operation of embodiments. It implements portions of a faultisolation mechanism whereby the state of operation and/or executionprocess of the various out-of-process renderer components 204 andnon-renderer components 202 may be monitored for inactivity, delay, orother anomalous operation. For the purposes of embodiments disclosedherein, anomalous operation may be appreciated as being any undesirableactivity, lack of activity, performance quality, temporary or permanentdegradation, or other aspect of functionality that includes, withoutlimitation, the operation of out-of-process renderer components 204 andnon-renderer components 202 operating within the scope of one or moresaid embodiments. The general functions of the watchdog module 240 inembodiments may variously include guarding against anomalous operation,of warning users of the occurrence of said anomalous operation, and ofreacting to any of potentially several types of anomalous operation.

In embodiments, the inactivity or delay of response may be specifiedwithin a parameter list within the watchdog module 240, or any othermodule within the client program 200 or external entity interacting withit 200. In a further embodiment, the watchdog module 240 may implementor otherwise rely upon one or more timers, for example, for each of thenon-renderer 202 and renderer 204 components present within anembodiment and/or which must be monitored. A further subcomponent, alsoconfigurable in some embodiments, may also allow a user to configure theprecise set of functional parameters, constraints, behaviors, andpolicies by which said watchdog module 240 must operate. Such a set ofparameters may include any or all of the non-renderer 202 and renderer204 components present within an embodiment, which may accordingly bemonitored for any one or more of inactivity or delay of response beyonda specified duration. In a further embodiment, such monitoring maylikewise include functionality to detect performance lags and in a stillfurther embodiment, performance surpluses.

Once the aforementioned specified duration elapses, the watchdog modulemay take any of various actions, whether corrective or terminal. Suchcorrective action may in an embodiment consist of restarting any one ormore of an out-of-process renderer 204 or non-renderer 202 componentfollowing any form of anomalous operation thereof. In a furtherembodiment, the watchdog module may be configured to automaticallyrestart any one or more of the latter components 202, 204 followingtheir anomalous operation. In another embodiment, the watchdog module240 may activate an alarm or instruct another available or otherwiseredundant non-renderer component 202 or renderer component 204 to takeover the load of the component having previously functioned anomalously.

In another embodiment, the watchdog module 240 may receive performancemonitoring information from non-renderer components 202 and/or renderercomponents 204. When provided with such information, a performanceanalysis submodule, typically functioning within the watchdog module 240may take precautionary measures to avoid the occurrence of anomalousoperation. Such precautionary measures may consist of validating theimpending occurrence anomalous operation to an instance of one oranother type of component 202, 204 and issue a command to initiate anautomatic failover to another redundant non-renderer 202 or renderercomponent 204. In a further embodiment, an intermediary state duringwhich a portion or the entirety of a component's state is temporarilysaved may be traversed in instants prior to said failover so as to causeminimal disruption in the operation as perceived by a human user of saidcomponent 202, 204 as a result of anomalous operation.

It will be appreciated that the watchdog module 240 may likewiseinitiate additional types of action, whether of a corrective or recoverynature, which may include performing operations upon other modules notnecessarily contained within the client program 200. In anotherembodiment, especially in a case in which corrective action is notpossible, such action may likewise additionally include initiating aprocessor reset. Likewise, the watchdog module 240 may, based on thedecision-making capabilities available to it, as a result of apreviously set policy, or further to user instruction, terminate any oneor more out-of-process renderer components 204 or non-renderercomponents 202.

It will be appreciated that said decision-making capabilities caninclude a set of one or more parameters configured to detect theoccurrence—or lack thereof—of any number of conditions indicating (orpotentially indicating) anomalous operation of one or more non-renderercomponent(s) 202 or out-of-process renderer component(s) 204.

In another embodiment, the watchdog module 240 may issue instructions tothe window module 260 (by way of the operations setup module 250) todirect the window module 260 to issue any of various notifications orprompts to users, as further described herein. In a further embodiment,and where logically appropriate, the watchdog module 240 may be furtherconfigured to implement the production of error logs or other exceptionhandling-related reporting for purposes of troubleshooting. Likewise, ina manner described previously herein, the watchdog module 240 caninstruct the window module 260 to present to one or more users a messagedescribing the occurrence of anomalous operation—or alternatively theoccurrence of a condition corresponding to at least one parameter withinthe aforementioned parameter set. Said message may further include aprompt or request that a user authorize the restarting and/or thetermination of said non-renderer 202 or renderer 204 component(s).

As discussed herein, the various prompts and notification messagespresented by the window module 260 may be any of several interrupttypes, such as maskable or non-maskable, the nature of which may in afurther embodiment depend on the severity of the specific exception toreport. Likewise, it will be appreciated that the type of interruptgenerated as a result of an instance of anomalous behavior may vary inaccordance with implementation-based considerations. In a non-limitingenumeration such interrupts may be a hardware and/or software type.Furthermore, individual interrupts may be edge-triggered,level-triggered, message-signaled, or any hybrid form of any knowninterrupt type. Such hybridization is particularly well suited if theexception to signal includes additional forensic and/or troubleshootinginformation on the events leading up to the anomalous operation of saidcomponent(s) 202, 204.

It will be further appreciated that the watchdog mechanisms describedherein are particularly valuable in detecting anomalous operations inthird party components, such as might be obtained under an open sourcelicense, or from an unknown party.

In its capacity as an operational component of embodiments describedherein, it will likewise be appreciated that the watchdog module 240 mayin some of said embodiments reside within the client program 200.Consistent with the out-of-process optimization sought in designs and inembodiments described herein, the watchdog module 240 may more optimallyreside outside the client program 200, and in at least some of suchembodiments may further execute within an external process, said processbeing a dedicated one. In a manner further discussed herein, theprocess, operation, data properties, or data assets that additionallyconstitute the watchdog module 240 need not—whether individually or as agroup—have an entirely internal or external disposition; in someembodiments any or all of the foregoing may be internal or external tothe client program 200.

Additional Observations

While a large number of embodiments deriving from the modules variouslydescribed herein have been non-limitingly proposed and described, itwill be appreciated that additional elements may be likewisecontemplated.

In a further embodiment, such elements may include the incorporation ofauthorization privileges or settings allowing specific users orcategories of users to operate any one or more modules or perform anyone or more specific actions.

In another embodiment, any part, module, or submodule of the clientprogram 200, or any portion external to it, may be implemented on anyone or more computers. Likewise, numerous network and other deploymenttopologies may be contemplated. In a related series of embodiments, theclient program 200 need neither be deployed nor operate contiguously orbe physically contained within a given site.

The functionality described for any module and/or embodiment herein maybe functionally implemented, in another embodiment, whether in whole orin part, within any other module whose functionality has (oralternatively has not) otherwise been described herein. Likewise, anyduplication or additional instantiation of any module, component orelement described herein may likewise be contemplated without departingfrom the essence of the various embodiments discussed hereinabove.

In addition, embodiments contemplated herein include variants, orportions thereof, contained upon any form or kind of tangible, computer-or machine-readable storage medium to implement any part of anystructure or for performance of any part or all operation for theembodiments described herein, or other embodiments related thereto. Sucha medium may in an embodiment be a virtual or cloud storage site. In afurther embodiment such a site may be secured with access permissions.

In a still further embodiment, any or all of the modules describedherein may allow one or more users to run an instantiated client sessionon a cloud or remote server, the client session enabling one or moreusers to perform any or all of the actions described for otherembodiments disclosed herein. In the course of said session, which in aseries of further embodiments may include the possible use of existingor proprietary communication and application protocols and serviceswhich provide support for communications occurring in real-time and overmultiple nodes, the server may send the client any or all of thecommands, message windows, popups, prompts, or other information asdisclosed in any other embodiment herein.

The above description has been provided for the purpose of illustrating,not limiting the invention which is defined by the appended claims.

1. A method of presenting a plurality of security camera feeds within asecurity video monitoring system for interaction by a user, said methodcomprising: executing a first program within an operating system of saidsecurity video monitoring system to run at least a first process forimplementing a semi-transparent window parent structure comprising anarray of transparent tiles, each transparent tile of said array oftransparent tiles having specific positions and dimensions within saidsemi-transparent window parent structure defined by user input throughsaid operating system; running a plurality of out-of-process renderercomponents in a plurality of processes external to the first process andunder said operating system of the security video monitoring system,said plurality of out-of-process renderer components each receiving acamera feed from one of said plurality of security camera feeds andproviding a video output resulting in a plurality of video outputs inseparate windows controlled by said operating system; positioning, anddimensioning each of said separate windows of said plurality of videooutputs within one tile of said array of transparent tiles according tosaid specific dimensions, wherein said separate windows of saidplurality of video outputs are each made to align behind a correspondingtransparent tile of said array of transparent tiles; wherein said firstprogram comprises a GUI playback interface element that collects userpoint/click input and in response thereto transmits a specificinstruction to one of said plurality of out-of-process renderercomponents.
 2. The method defined in claim 1, further comprising causingvisual elements of one or more non-renderer components, saidnon-renderer components operating out-of-process, to be displayedsuperjacent to said visual outputs and subjacent to said tiles.
 3. Themethod defined in claim 1, further comprising accepting user input forone or more specific positions within said window parent structure anddispatching corresponding commands to out-of-process components at saidone or more specific positions.
 4. The method defined in claim 1,wherein said user point/click input is any one or more from the groupcomprising: a mouse click, a drag-and-drop action, a tactile action orgesture, a three-dimensional action or gesture, a key press, and asensory input.
 5. The method defined in claim 4 wherein said sensoryinput is received via one or more voice commands.
 6. The method definedin claim 4, wherein said sensory input is received via eye control. 7.The method defined in claim 4, wherein said sensory input is receivedfrom of gesture recognition via one or more multimodal sensors.
 8. Themethod defined in claim 1, further comprising monitoring the executionstatus of said out-of-process renderer components and restarting any oneor more said out-of-process renderer components following anomalousoperation thereof.
 9. The method defined in claim 1, further comprisingmonitoring the execution status of said out-of-process renderercomponents and terminating any one or more said out-of-process renderercomponents following anomalous operation thereof.
 10. The method definedin claim 8, wherein said monitoring includes using a condition parameterset to define or to detect the occurrence of said anomalous operation.11. The method defined in claim 10, wherein said condition parameter setcomprises a member specifying the minimum time interval to wait betweensaid anomalous operation and said restarting.
 12. The method defined inclaim 11, further comprising notifying one or more human users of saidanomalous operation by way of a dialog box.
 13. The method defined inclaim 12, further comprising presenting to one or more human users oneor more decisions on how to proceed following said anomalous operation.14. A non-transitory computer-readable storage medium for implementing asecurity video monitoring client for presenting a plurality of securitycamera feeds, said medium comprising instructions for causing aprocessing device to perform a method comprising: executing a firstprogram within an operating system of said security video monitoringsystem to run at least a first process for implementing asemi-transparent window parent structure comprising an array oftransparent tiles, each transparent tile of said array of transparenttiles having specific positions and dimensions within saidsemi-transparent window parent structure defined by user input throughsaid operating system; running a plurality of out-of-process renderercomponents in a plurality of processes external to the first process andunder said operating system of the security video monitoring system,said plurality of out-of-process renderer components each receiving acamera feed from one of said plurality of security camera feeds andproviding a video output resulting in a plurality of video outputs inseparate windows controlled by said operating system; positioning, anddimensioning each of said separate windows of said plurality of videooutputs within one tile of said array of transparent tiles according tosaid specific dimensions, wherein said separate windows of saidplurality of video outputs are each made to align behind a correspondingtransparent tile of said array of transparent tiles; wherein said firstprogram comprises a GUI playback interface element that collects userpoint/click input and in response thereto transmits a specificinstruction to one of said plurality of out-of-process renderercomponents.
 15. The medium defined in claim 14, further comprisinginstructions for causing a processing device to display visual elementsof one or more non-renderer components, said non-renderer componentsoperating out-of-process, being situated superjacent to said visualoutputs and subjacent to said tiles.
 16. The medium defined in claim 15,further comprising instructions for causing a processing device tooverlay, as said visual elements, any one or more from among the groupcomprising software control component objects, interface components,digital on-screen graphics, and motion graphics over a respective visualoutput.
 17. The medium defined in claim 14, further comprisinginstructions for causing a processing device to accept user input forone or more specific positions within said window parent structure andto dispatch corresponding commands to out-of-process components at saidone or more specific positions.
 18. The medium defined in claim 17,further comprising instructions for causing a processing device toreceive user input to scale ones of said visual elements in proportionto a change in dimension of said window parent.
 19. The medium definedin claim 14, further comprising instructions for causing a processingdevice to monitor the execution status of said out-of-process renderercomponents and to restart any one or more said out-of-process renderercomponents following anomalous operation thereof.
 20. The medium definedin claim 14, further comprising instructions for causing a processingdevice to monitor the execution status of said out-of-process renderercomponents and to terminate any one or more said out-of-process renderercomponents following anomalous operation thereof.